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Abstract 



•his chapter addresses different modalities of assessment, provides an 


A overview of the assessment processes of the basic skills (reading, writing, 
listening and speaking) tested by the Cambridge First Certificate in English (FCE) 
examination, and explores the ways in which the FCE Online Course and Tester 
-the resources provided through the InGenio CALL authoring shell, content 
manager and courseware delivery platform- contribute to the effectiveness and 
efficiency of the assessment of those skills. These materials are flexible enough 
to be adapted to different degrees of learning autonomy, thus allowing students to 
assess their own progress while enabling their learning process to be monitored 
by a tutor or a teacher in those cases where the materials are used by them as 
a complement for their language lessons. Such is the case of some teachers at 
the Universidad Politecnica de Valencia (UPV) who are already successfully 
making use of some of the materials published through InGenio as a way to 
include additional contents and monitor and assess their students’ achievements 
and progress. One of the most important features concerning these online 
language learning materials is that they allow students to choose between two 
main modalities of assessment, one of them enabling them to supervise their own 
learning process; and the other one leading to the development of their autonomy 
and sense of responsibility while getting support from a tutor or teacher. 
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Chapter 4 


1. Introduction 

The importance of assessment is derived from both its influence in the way teachers 
and students address language teaching and learning and the changes it might 
introduce as far as methodologies, approaches and behaviour of all participants in 
the language learning process are concerned. Brown and Glasner (2003) consider 
assessment as one of the key aspects in education and define it as the “dynamic 
developmental process which develops and changes as the needs arise and as 
understanding of the process improves” (preface). The washback effect, defined 
by Messick (1996) as “the extent to which the introduction and use of a test 
influences language teachers and learners to do things they would not otherwise 
do that promote or inhibit language learning” (p. 241), is a frequent phenomenon 
which should also be taken into consideration when dealing with the process of 
assessment. It is usually found in the language classroom in the form of innovations 
and new ways of teaching in an attempt to foster students’ motivation and the 
emergence of a greater variety of learning strategies. Teachers usually try to adopt 
new methodologies and implement new approaches to language teaching as an effort 
to adapt the contents and materials they develop to meet the students’ individual and 
specific needs, goals, interests and expectations. In this particular case, not only do 
the students who learn and practice by using the FCE Online Course and who assess 
their knowledge through the FCE Online Tester try to attain the target language 
level, but they also aim at passing the official examination. Exam criteria were 
taken into consideration by the authors of these preparatory materials which were 
thus adapted to such demands and to the typologies of exercises comprised by the 
official examination. This fact could be seen as an example of how the washback 
effect has been present in the process of material design and development. A wide 
section of exercises included in the course and tester follow the typology, structure 
and level of the exam. Therefore, students are allowed to practice and assess their 
knowledge in accordance with the final exam criteria. 

This chapter explores some of the key issues concerning the assessment process 
of skills tested by the Cambridge First Certificate in English Examination* and 


““Further information available at Cambridge ESOL webpage: http://www.cambridgeesol.org/exams/general-english/fce.html 
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the ways in which the InGenio* online preparatory materials contribute to its 
effectiveness and efficiency in both self-assessment and tutor assessment, with 
a special focus on the two most recently developed learning materials within 
the CAMILLE Research Group** at the UPV: FCE Online Course and FCE 
Online Tester. InGenio is a content manager tool which allows users to develop 
preparatory materials and activities through a system based on templates. One of 
the main advantages of the materials created is their adaptability and flexibility. 
It allows students to have access to two different modalities of assessment and 
to two different learning options, thus enabling a wider range of students with 
different characteristics and needs to organise the way in which they intend to 
face the learning process in the most convenient way, for each of them, so as 
to obtain better results. Students’ self-assessment is the first learning modality 
offered. It enables students to conduct their own learning process and to assess 
their own learning achievements in an independent, and autonomous way. 
Tutor assessment, the other modality provided by these materials, leads to the 
development of the students’ autonomy and sense of responsibility over the 
learning process, while enabling students to get as much help and support as 
they need from a human tutor in order to both complete the different tasks and 
attain the target level of language. 


2. The skills tested by the FCE examination 

Assessment is a very important component of the learning process. Its 
importance mainly derives from the fact that it is capable of influencing the 
way teaching and learning are addressed. Assessment is likely to make teachers 
and learners, who are the main participants in the learning process, introduce 
changes in their methodologies, approaches, behaviour and learning strategies. 
These changes would be based on “the particular kind of knowledge or ability 
that a test is designed to measure” (Read, 2000, p. 95), that is, the construct, a 

* InGenio is a free online content delivery and management platform with a number of language courses available. Among them 
are: Intermediate Online English, Valencia Interactiu - Grau Mitja, and beginners and elementary courses for learners of Czech and 
Slovak. The system has been developed by the CAMILLE Research Group led by Dr. Ana Gimeno-Sanz (Department of Applied 
Linguistics - Universidad Politecnica de Valencia). See http://camilleweb.upv.es/camille for further information. 

** CAMILLE stands for Computer Assisted Multimedia Interactive Language Learning Environment 
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concept which is closely linked to the process of content validity. Fulcher and 
Davidson (2007) define content validity as “any attempt to show that the content 
of a test is a representative sample from the domain that is to be tested” (p. 6). 
Buck (2001) considers that the essential condition for any test to be acceptable 
is that it measures the appropriate construct, this being the only possible way to 
ensure validity and usefulness of that assessment. In order to design the right test 
to measure the four main skills tested by FCE, i.e., reading, writing, listening 
and speaking, it is important to know the micro-skills and strategies that the 
candidates need to put into practice in order to ascertain that they have the 
appropriate level, as well as the way to assess those skills. 

When dealing with the assessment of the reading skills, several guidelines about 
teaching and curriculum planning, mentioned by authors such as Grabe (1991, 
2008, p. 81), should be taken into consideration. Reading should be integrated 
together with other skills within content; the texts should be interesting and 
related to the candidates’ education, hobbies and particular interests; the 
different reading sub-skills should be measured; silent reading should be 
fostered, and so should reading comprehension. Furthermore, the person in 
charge of the assessment should be able to accept different interpretations of the 
texts, organising skills and strategies systematically and taking into account the 
characteristics and objectives of each group. 

According to Alderson (2000), the reading assessment guidelines also apply to the 
different levels of understanding to be assessed, which are “literal understanding 
of a text, an understanding of meanings that are not directly stated in text, or an 
understanding of the main implications of a text” (p. 7), as well as the distinction 
between what Gray (1960, in Alderson, 2000, p. 7) called reading “the lines”, 
that is, understanding the literal meaning of a text, reading “between the lines” or 
inferring meanings, and reading “beyond the lines”. This means making critical 
evaluations and judgements of a text. Because there are so many aspects to take 
into account when assessing reading, as Alderson (2000) points out, it is very 
important to find a way to measure reading accurately, considering to what extent 
the tests reflect and are based on previous research and literature referred not only 
to the process but also to the product. When trying to find the appropriate way to 
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measure reading comprehension, one of the main things to consider is what the 
relevant task characteristics are. These can be determined with the help of the 
following framework (Table 1), designed by Bachman and Palmer (1996). 


Table 1. Framework of task characteristics 

Adapted from Bachman and Palmer (1996). 


Setting 

Physical circumstances under which either language use or 
language testing takes place: 

• physical characteristics of the setting; 

• participants involved; 

• time of the task. 

Rubric 

Context for the task: 

Those characteristics that provide the structure for the 
task and constrain how language users or test takers are 
expected to respond to these tasks. 

Input 

Material contained in the task, which test takers need to process 
in some way, and to which they are expected to respond. 

Expected response 

Language use that is expected, given the way in which the 
rubric, or context, for the task is configured, and the particular 
input that is provided. 

Relationship between 
input and response 

• Reactivity: degree of reciprocity, or interaction involved. 

• Scope: amount and range of input that needs to be 
processed in order to respond. 

• Directness: extent to which the response can be made 
by using information in the input by itself, or whether the 
language user or test taker must also rely on information 
in the context or in his or her own real world knowledge.. 


As far as the use of computers for reading is concerned, it is advantageous 
for learners because of several features such as the instantaneous access it 
provides to many kinds of authentic and communicative reading materials 
through the Web, the capacity to add hypertexts in order to access other texts 
or useful links as well as additional explanations on vocabulary and grammar, 
the addition of multimedia files as a complement of the reading materials, and 
the capacity to control reading speed and other aspects of learning (Levy & 
Stockwell, 2006). In addition, the Web is a rich source of written materials that 
can contribute to both the improvement of reading skills to higher levels and 
the students’ cultural competence (Taylor & Gitsaki, 2004). Nevertheless, there 
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is also a series of disadvantages associated with reading on the computer and 
on the Web. One such disadvantage is the fact that it might be uncomfortable, 
although this would probably not be an inconvenience if other kind of devices, 
such as iPads, or more recent developments in screen technology were used 
instead. Another inconvenience is the fact that it might be harder to find 
the appropriate texts or excerpts among the huge quantity of materials and 
information available. In addition, it might be hard to pick the materials with 
the appropriate level, both linguistically and socially. However, being aware 
of these drawbacks can help a teacher face them with ease so that a great 
quantity of useful materials can be used, integrated and adapted to the context 
of language learning (Dudeney, 2000). 

The materials developed to be implemented through the InGenio online platform 
aim to offer resources to practice the four basic linguistic skills. Concerning 
reading, InGenio provides the authors with several exercise templates that allow 
them to create a varied range of activities for autonomous practice and self- 
assessment. Some of these templates are: reordering, matching, monitoring 
comprehension, multiple-choice questions (single selection with pull-down 
menu, single selection menu, and multiple selection), gap-filling exercises, 
reinforcing new vocabulary, vocabidary building, and word search puzzles. The 
contents included in the tasks and texts that are part of the exercises are not only 
devoted to practicing and assessing reading skills but also to helping students 
think critically and analytically and have a justified or well-founded opinion 
about some of the world’s most important or interesting facts and events. Learners 
are also provided with more resources such as reference materials, additional 
explanations and hints, further reading, appropriate feedback according to the 
student’s performance, automatic communication of residts through progress 
reports, and printing-enabled screens. 

As for writing, historically it tended to be considered more prestigious and elitist 
than speaking (Brown & Yule, 1983; Carter & McCarthy, 2006; Gilmore, 2007). 
Nowadays, it is an essential tool of communication in the global community we 
are living in. Additionally, there has been a change in the role of writing as it has 
shifted from “conveying information” to “transforming knowledge to create new 
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knowledge” while helping to predict future professional and academic success. 
The importance of writing as a predictor might explain the great demand of valid 
and reliable ways to test writing ability (Weigle, 2009). According to Hamp- 
Lyons (1991), there are two main ways to assess writing: direct and indirect. 
The main characteristics of a “direct” test are as follows (Hamp-Lyons, 1991): 
candidates must write at least one piece of continuous text, they are given a set of 
instructions or “prompts” but have some freedom in their responses, each test is 
usually read by more than one trained rater, judgments are tied to a set of sample 
responses or rating scales, and these judgments are expressed as numbers. In 
addition, there are other important characteristics such as the limited time frame, 
generally between thirty minutes and two hours, and the fact that the topic is 
unknown to test takers in advance (Weigle, 2009, p. 59). The dimensions of 
tasks for direct writing assessment are specified in Table 2. As for the “indirect” 
tests of writing or “timed impromptu writing tests”, they most often consist of 
multiple-choice tests of grammar and usage (Hamp-Lyons, 1991). 


Table 2. Dimensions of tasks for direct writing assessment 

Adapted from Purves, Soter, Takala, and Vahapassi (1984, pp. 397-398), 
and Hale (1996), cited in Weigle (2009, p. 63). 


Dimension 

Examples 

Subject matter 

Self, family, school, technology, etc. 

Stimulus 

Text, multiple texts, graph, table 

Genre 

Essay, letter, informal letter, informal note, advertisement 

Rhetorical task 

Narration, description, exposition, argument 

Pattern of exposition 

Process, comparison/contrast, cause/effect, classification, definition 

Cognitive demands 

Reproduce facts/ideas, organise/reorganise information, 
apply/analyse/synthesise/evaluate 

Specification of: 

• Audience 

• Role 

• Tone, style 

• Self, teacher, classmates, general public 

• Self/detached observer, other/assumed persona 

• Formal, informal 

Length 

Less than 'A page, 'A to 1 page, 2-5 pages 

Time allowed 

Less than 30 minutes, 30-59 minutes, 1-2 hours 

Prompt wording 

Question vs. statement, implicit vs. explicit, amount of context provided 

Choice of prompts 

Choice vs. no choice 

Transcription mode 

Handwritten vs. word-processed 

Scoring criteria 

Primarily content and organisation; primarily linguistic accuracy; 
unspecified 
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As far as the process of test development is concerned, it occurs in three main 
stages: design, operationalisation and administration (Bachman & Palmer, 1996). 
These stages are followed by a very important procedure: scoring, which is used 
in making decisions and inferences about the performance of the exam takers and 
therefore must be accurate and derive from appropriate, theoretically-grounded 
and consistent rating scales and scoring rubrics (Weigle, 2009, p. 108). Writing 
assessment has overcome dramatic changes due to the impact of technology 
and increased global communication. In fact, the nature of writing itself has 
been affected by ICT “in terms of process, norms and standards” (Weigle, 2009, 
p. 231) and the emergence of scoring of writing by computer is picturing the 
future of computers as supplements of human raters, especially in the case of 
large-scale writing assessments such as FCE. 

Creativity has a dominant role in the practice of writing, as well as in the practice 
of speaking. Concerning the InGenio FCE online materials, in the sections 
devoted to practicing writing and speaking, more freedom is given to students 
in order to allow them to write and speak about topics that concern them or 
that they find interesting and attractive. The tasks which are being designed fit 
the official FCE exam criteria, and try to present these training sections in a 
more innovative way through more open and reflective approaches. The students 
also have access to some useful recommendations in order to finally respect 
the limits fixed by the construct tested by the official exam. Moreover, course 
designers may need to take into account the type of texts that learners need to 
write and read in their L2, which could markedly differ from the texts that they 
write or read in their LI (Ferguson, 2007). Open input without sound and open- 
input with sound are among the most useful InGenio templates when it comes 
to encouraging students “to analyse data and subsequently reason their answers” 
(Gimeno-Sanz, 2009a, p. 93) and write their own input in the form of rewriting, 
information transfer, short answers, etc. 

As for listening, it is considered as the least understood of all skills (Alderson 
& Bachman, editors’ preface, in Buck, 2001) in spite of being the most 
important one because of its potential influence in teaching methodologies. 
It is also difficult to measure at a technical level, and it is time-consuming 
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too, which may explain the frequent reluctance to test it. Nevertheless, it 
is essential to give listening assessment the importance it deserves, given 
that the consequent washback could influence teachers and make them 
aware of the fact that developing listening skills is crucial for students to 
be able to communicate in the target language. This is especially true in 
those contexts where the students’ L2 is the vehicular language and therefore 
the main means of interaction. The following listening framework describes 
its two main components: language competence and strategic competence. 
Each competence implies different kinds of knowledge, as shown in Table 3 
below, adapted from Buck (2001, p. 104). 


Table 3. Framework for describing listening ability 

Adapted from Buck (2001, p. 104). 


Language competence 

Knowledge about language that the listener brings to the 
listening situation. Includes fully automated procedural 
knowledge and controlled or conscious declarative 
knowledge: grammatical, discourse, pragmatic, and 
sociolinguistic knowledge. 

Grammatical knowledge 

Understanding short utterances on a literal semantic level. 
Includes phonology, stress, intonation, spoken vocabulary, 
spoken syntax. 

Discourse knowledge 

Understanding longer utterances or interactive discourse 
between two or more speakers. Includes knowledge 
of the discourse features (cohesion, foregrounding, 
rhetorical schemata and story grammars) and knowledge 
of the structure of unplanned discourse. 

Pragmatic knowledge 

Understanding the function or the elocutionary force of 
an utterance or longer text, and interpreting the intended 
meaning in terms of that. Includes understanding 
whether utterances are intended to convey ideas, 
manipulate, learn or are for creative expression, as well 
as understanding indirect speech acts and pragmatic 
implications. 

Sociolinguistic knowledge 

Understanding the language of particular sociocultural 
settings, and interpreting utterances in terms of the 
context of situation. Includes knowledge of appropriate 
linguistic forms and conventions characteristic of 
particular sociolinguistic groups, and the implications 
of their use, or non-use, such as slang, idiomatic 
expressions, dialects, cultural references, figures of 
speech, levels of formality and registers. 
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Strategic competence 

Includes cognitive and metacognitive strategies, or 
executive processes, that fulfil the cognitive management 
function in listening. It is the ability to use language 
competence, and includes all the compensatory strategies 
used by second language listeners. 

Cognitive strategies 

Mental activities related to comprehending and storing 
input in working memory or long-term memory for later 
retrieval. 

Comprehension 

processes 

Associated with processing of linguistic and non-linguistic 
input. 

Storing and memory 
processes 

Associated with storing of linguistic and non-linguistic 
input in working memory or long-term memory. 

Using and retrieval 
processes 

Associated with accessing memory, to be ready for 
output. 

Metacognitive strategies 

Conscious or unconscious mental activities that perform 
an executive function in the management of cognitive 
strategies. 

Assessing the situation 

Taking stock of conditions surrounding a language task by 
assessing one’s own knowledge, one’s available internal 
and external resources and constraints of the situation 
before engaging in the task. 

Monitoring 

Determining the effectiveness of one’s own or another’s 
performance while engaged in a task. 

Self-evaluating 

Determining effectiveness of one’s own or another’s 
performance after engaging in the task. 

Self-testing 

Testing oneself to determine the effectiveness of one's 
own language use or the lack thereof. 


Out of the three main approaches to assessing listening, that is, discrete-point, 
integrative and communicative, Buck (2001) considers the communicative 
approach, characterised by using the language “for the purpose of 
communication, in a particular situation and for a particular purpose” (p. 83), 
the most effective one. In this approach, the texts to be listened to are 
authentic and genuine. There is a well-defined goal and the examinees have to 
accomplish authentic tasks. The basic idea underlying communicative testing 
is that these tests emulate the use of language in the real world, which means 
that the construct to be assessed is richer and more realistic, in such a way 
that it “allows the examiner to make useful inferences about the examinee’s 
communicative abilities” (Buck, 2001, p. 92). As for the use of technologies 
in assessing listening, Levy and Stockwell (2006) underline that they should 
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be used in such a way that they provide something extra that was not available 
through traditional teaching and assessment, in order to enhance learning. The 
success of multimedia tasks, according to Hoven (1999), depends on the way 
they are designed as far as individual differences among students, learning 
styles and preferred learning models are concerned. 

The InGenio preparatory materials are not devoted uniquely to practicing 
through written channels and static texts, but to expand oral and communicative 
skills by introducing tasks and scenarios aimed at fostering real practice of oral 
skills. This is done through corpora of audio excerpts (listening) and voice 
recording systems (speaking) which can be uploaded onto the system for a 
tutor to assess and subsequently provide appropriate feedback, in the case 
of completely autonomous practice and assessment, students are provided 
with models so as to allow them to compare them with their own answers. 
In particular, concerning the listening skills, the InGenio content manager 
provides editors with video and audio templates as well as with the means to 
build up an abundant pool of images and audio and video files. Many of the 
templates used to practice and assess reading skills could also be used for the 
listening activities. 

Finally, in relation to the assessment of speaking skills, numerous studies 
point out that this is one of the most complex and controversial aspects 
within second language teaching (Weir, O’Sullivan, & Horai, 2006). This 
is due to the difficulties encountered when trying to join the targets of 
the assessment and the appropriate tasks or instruments that assessment 
requires (Luoma, 2009). Moreover, speaking is considered the hardest skill 
to be taught and tested through computers, which might explain the “lack 
of representativeness” of studies focusing on speaking (Levy & Stockwell, 
2006, p. 181). This makes the integration of assessment of communicative 
speaking into CALL materials a true challenge. Luoma (2009) conceives 
speaking assessment as a cycle in which the participants involved are the 
examinees, the interlocutors, the examiners, and the rating/marking criteria. 
The following illustration (Figure 1), adapted from Luoma (2009, p. 5), 
outlines the oral assessment activity cycle. 
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Figure 1. Oral assessment activity cycle 

Adapted from Luoma (2009, p. 5). 



Administration / 
Performance 


As this graph shows, the cycle starts when the necessity of speaking assessment 
is perceived; planning and development stages follow, resulting in the definition 
of the construct. Next, the criteria are determined as well as the way in which 
the exam is to be administered. Assessment then takes place by means of two 
interaction processes: firstly, exam administration and candidates’ performance 
in interactions (among the candidates and/or the examiner) in which they show 
their oral production skills; and secondly, a rating process in which the examiners 
apply the assessment criteria to the candidates’ performance in order to obtain a 
grade for each of the candidates. 

Prior to assessing speaking, the type of speech to be assessed has to be 
determined: planned or not, formal or informal, etc., as there might be substantial 
changes in vocabulary choices, grammar constructions or pronunciation 
depending on the type of discourse (Luoma, 2009). Another factor that could 
influence the type of speech acts is the social/situational context, which can be 
determined by using Hymes’ (1974) SPEAKING model when planning and 
describing the construct (Table 4). 
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Table 4. SPEAKING Model 

Adapted from Hymes (1974). 


Setting/Scene 

“Setting refers to the time and place of a speech act and, in 
general, to the physical circumstances” (Hymes, 1974, p. 55). 

Scene is the “psychological setting” or “cultural definition” of a 
scene, including characteristics such as range of formality and 
sense of play or seriousness (Hymes, 1974, pp. 55-56). 

Participants 

Speaker and audience. Linguists will make distinctions within 
these categories; for example, the audience can be distinguished 
as addressees and other hearers (Hymes, 1974, pp. 54-56). 

Ends 

Purposes, goals, and outcomes (Hymes, 1974, pp. 56-57). 

Act Sequence 

Form and order of the event. 

Key 

Cues that establish the “tone, manner, or spirit” of the speech act 

(Hymes, 1974, p. 57). 

Instrumentalities 

Forms and styles of speech (Hymes, 1974, pp. 58-60). 

Norms 

Social rules governing the event and the participants' actions and 
reaction. 

Genre 

The kind of speech act or event. 


Context is the first element Hymes (1974) mentions. It concerns all the aspects 
and elements that are present at the moment of interaction, i.e., “the linguistic, 
physic, psychological and social dimensions of the language that is being used” 
(Luoma, 2009, p. 30). Context also refers to the concrete aspects of the situation in 
which the interaction takes place, such as the place or the interlocutors’ previous 
experiences of use, what is said and the way it is said in different situations. 
The implications are that conversation can be led by task designers to a certain 
extent through the manipulation of the context characteristics by means of the 
proprieties that are attributed to the tasks and to the activities suggested (Luoma, 
2009, p. 30). Nevertheless, conversations cannot be predicted in a very precise 
manner, as Douglas (1998) points out, because the interlocutors are likely to 
be influenced by the context features considered by them as the most salient 
and therefore there might be variation depending on the individuals and on the 
situations involved in the process. 

The integration of speaking tasks in FCE Online Course and Tester is an 
innovation, since speaking skills are not practised nor tested by most previously- 
existent self-assessment preparatory FCE materials. These materials tend to 
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give advice about how to prepare for the exam and how to face it, as well as 
information about its structure and some other details, but they do not allow the 
candidates to practice and to assess their speaking abilities. Nevertheless, this 
feature had already been predicted by Gimeno-Sanz (2009b), who foresaw the 
design of speaking tasks through voice recording in the InGenio online content 
delivery and management platform which would then add up to the already 
implemented features: 

Although voice recording has not been programmed into the system, it is 
nevertheless possible to design exercises where the learner is requested to 
record his or her own utterances by accessing the Windows Media Player, 
which can be called up and minimised when not in use. Evaluation of oral 
production will be subject to learner comparison with a pre-recorded model 
or by tutor intervention. Future developments include incorporating voice 
recognition software into the InGenio system (Gimeno-Sanz, 2009b, p. 94). 

Apart from being an innovative aspect, the inclusion of speaking tasks in these 
preparatory materials would also be an advantage of the online format of the 
FCE Course and Tester. The lack of flexibility of the printed versions makes the 
assessment of the speaking ability an unfeasible task whereas the characteristics 
of the electronic format that benefits from the recent advances in technology 
facilitate this kind of assessment. In fact, there are many other advantages about 
the use of technologies such as computers for the assessment of speaking. One 
of these advantages is the reliability of the assessment, which can take the form 
of semi-assisted interviews recording the answers of the examinees to a set of 
questions, the performance of the candidate thus being safely saved and stored, 
ready to be subsequently assessed by a qualified rater. Another positive aspect is 
the authenticity of the tasks based on audiovisual repertoires. In addition, the use 
of 1CT can provide a high degree of interactivity, and very often it has a positive 
impact and washback effect in the classroom. Furthermore the practicality, 
for both the examiner and the examinee, of the use of a system based on 
computational linguistics together with the possibility to assess speaking more 
objectively are two additional advantages of the use of technology. In fact, the 
use of e-tools such as voice recording systems allows for the collection of data 
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while facilitating the evaluation of these data and the possibility to access them 
in case a revision of the results is needed. 


3. Self-assessment and tutor assessment 
in FCE Online Course and Tester 


The modalities of assessment offered by the materials of the InGenio authoring 
tool and learning management system (Figure 2) developed by the CAMILLE 
Research Group at the Universidad Politecnica de Valencia, such as FCE Online 
Course and Tester, are self-assessment and tutor assessment. 

Figure 2. The InGenio authoring tool homepage 
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The courses and materials published through the content manager platform 
InGenio allow users to follow the self-assessment modality. It includes a 
greater variety of materials and activities so as to cater for the needs of learners 
with different learning styles and preferences, thus enabling them to use the 
materials in the most convenient way. These materials include a greater number 
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of reference materials (Figure 3), additional explanations, extra readings and 
extra self-assessment activities in order to help those students who are mainly 
learning with less or no support from a human tutor. These features have the 
advantage of enabling students to conduct their own learning process and to 
evaluate their own learning achievements in an independent, autonomous and 
individual way. 


Figure 3. InGenio Editor’s interface for reference materials 
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InGenio also provides two modalities of activities: automatically corrected and 
tutorised activities. On the one hand, the users can find automatically corrected 
activities, since once the student has completed an exercise or task, the system 
offers immediate feedback by using the information provided by the authors 
of the exercises (correct answers and specific feedback). In this case, students 
receive automatically generated messages in response to the number of correct 
or incorrect answers. On the other hand, tutorised activities are those which 
require human intervention by providing personalised feedback programmed or 
provided by the author of the exercise. This is the case in some of the exercises 
dealing with oral and written production, where responses are generally more 
elaborate or cannot be limited in order to give students automatic feedback. 
The following examples (taken from Gimeno-Sanz, 2009b) show the template 
where content designers can introduce their feedback (Figure 4), the automatic 
feedback by items in the way students see it (Figure 5), and the general feedback 
measured by rate of efficiency (Figure 6). 
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Figure 4. Author feedback form in InGenio 



Figure 5. Feedback by item 
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Figure 6. Feedback measured by rate of efficiency 
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Feedback, obviously, is a key component of these materials as “it should 
always be clear what kind of mistake has been made”, and it is a useful means 
to “provide not only awareness as to where the mistake lies, but also how to 
improve the learner’s performance” (Gimeno-Sanz, 2009a, p. 88). Moreover, 
it should be carefully thought through and implemented when writing self- 
access materials. Abrupt statements such as “No”, “Incorrect, try again”, 
etc. must be avoided (Gimeno-Sanz, 2009a, p. 88). In order to provide the 
most suitable kind of feedback, content providers should try to anticipate 
and predict learners’ behaviour and reactions when completing exercises or 
activities which are part of self-access materials. These predictions could 
be based on a corpus of the common mistakes made by Spanish speakers in 
official B2 examinations that is currently being developed. The information 
about these mistakes has mainly been taken from the literature based on or 
informed by corpora such as the Cambridge Learner Corpus (CLC)*, but 

* Further information available at http://www.cambridge.org/es/elt/?site_locale=es_ES. 
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further improvements will include analysing the Universidad Politecnica de 
Valencia students’ results in the exam simulations and enriching the InGenio 
corpus with information about the mistakes the students made. 

As for the materials designed for the tutor assessment modality, they provide 
more support to be used in the classroom context and include resources such 
as materials specifically designed to provide support for the tutor, a teachers’ 
guide, and also detailed performance reports (Figure 7) about the students and 
other tracking devices (Gimeno-Sanz, 2008, p. 54). The positive aspect of this 
modality is the fact that it helps students to become autonomous learners (see 
Sevilla-Pavon, Martinez-Saez, & Gimeno-Sanz, in press), by fostering their 
sense of responsibility in the learning process while providing students with 
help, guidance and support. 


Figure 7. InGenio student’s assessment report 
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InGenio FCE Online Course and Tester also give students access to test 
simulations, similar to the actual FCE examination in terms of level, structure, 


63 

























Chapter 4 


exercise typology and administration mode*. The current development work of 
the CAMILLE Research Group is now focused on relating and adapting the 
content ofthese materials to the scientific and technical context of the Universidad 
Politecnica de Valencia. This is being done with a view to enriching the students’ 
specific knowledge (e.g., their technical and scientific vocabulary) so as to help 
them become better professionals in the future. At the same time, these materials 
aim to enable the students to prepare for an examination that would certify their 
B2 level of English in accordance with the Common European Framework of 
Reference for Languages (CEFRL), a new requirement to earn their degree 
according to the guidelines fixed by the UPV as part of the implementation of 
the Bologna Process. 

The self-assessment exercises and simulations included in the FCE Online 
Course and Tester benefit from numerous advantages of ICT, such as reliability, 
interactivity, authenticity, practicality, objectivity, and quick collection/evaluation 
of data. In this way, students are given the opportunity to face a similar situation 
to that of the FCE examination. This in turn helps them be aware of their strengths 
and weaknesses so that they can still work hard on those particular aspects they 
need to improve before facing the actual test. By doing so, students are likely to 
get better results not only linguistically but also in affective terms, since being able 
to improve and even to predict their outcomes before taking the test can foster 
their self-confidence and motivation while reducing their anxiety levels. These 
materials also allow designers to generate online assessment elements that provide 
valuable information about the students, available for teachers at any point so 
that they can observe and assess adequately the progress of every student. This 
is particularly useful in those cases in which these materials are used not only in 
preparation for the FCE examination but also as learning and assessment tools 
specifically oriented to technical and scientific languages learning. 

The FCE Online Course and Tester develop their full potential when used in 
combination, the first of these preparatory resources being an online course 


*These simulations are computer-based and therefore their administration mode is the same as that of the computer-based FCE, the 
computerised version of the FCE examination which was launched in January 2010 (further information available at http://www. 
cambridgeesol.org/exams/exams-info/computer-based-testing.html). 
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with different kinds of exercises -similar to the ones included in the FCE 
papers-to be completed first; and an assessment program including self- 
assessment exercises and simulations of the actual online examination, to 
be completed once the online course is finished and the students feel ready 
to face an examination situation. Prior to the design of simulations, a great 
number of exercises had to be compiled in a corpus of B2 exercises accessible 
through an online database. These exercises are either independent or based 
on given texts, the latter being more abundant than the former since texts 
are also predominant in the actual FCE examination papers. The simulations 
can be generated upon the users’ request, the great number of texts and 
exercises included in the database allowing for numerous and randomised 
combinations. 

In the development of these preparatory materials, the two main previous 
aspects underscored by Buck (2001) have been taken into account. First of all, 
the ultimate objective, which would be to succeed in the FCE examination and 
thus to ascertain the B2 level of English of our students, follows the application 
of the guidelines of the Bologna Process and of the Action Plan on Language 
Learning and Linguistic Diversity*. Then, the profile of the potential examinees 
is considered, which in our case would be that of students of technical and 
scientific degrees at UPV aged between 18 and 25, who need to prove their B2 
level of English -one of the means fixed by the UPV to do so is a success in 
the FCE official exam. As for the successful put into practice, Messick’s (1996) 
advice to avoid construct-underrepresentation and construct-irrelevant variance 
is also being followed. On the one hand, construct-underrepresentation occurs 
when “the test is too narrow and fails to include important dimensions or facets 
of the construct” (Messick, 1989, p. 34). On the other hand, construct-irrelevant 
variance (which is also called surplus construct irrelevancy) takes place when 
“the test contains excess reliable variance that is irrelevant to the interpreted 
construct” (Messick, 1989, p. 34), i.e., when the test assesses abilities which 
are not included in the theoretical description of the construct. Avoiding these 
inconveniences is a sine qua non requirement in assessment because the correct 

*Further information can be found at http://europa.eu/legislation_suinmaries/education_training_youth/lifelong_learning/ 
cl 1068 en.htm. 
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or incorrect interpretation of the scores, which in turn influences the validity of 
the decisions we make based on those scores, depends on the theoretical and 
operational soundness of our construct. 


4. Conclusion 

The InGenio FCE Online Course and Tester are good examples of the way in 
which the use of 1CT in language learning can contribute to the effectiveness 
and efficiency of the assessment process of basic skills such as reading, writing, 
listening and even speaking in two main modalities: student self-assessment 
and tutor assessment. The students are allowed to choose between these 
two different modalities of assessment on account of the flexibility of these 
materials. Their choice depends on their needs, preferences, learning styles and 
personal characteristics. This is possible thanks to the fact that the students can 
choose whether to use these materials as part of a blended-learning course (i.e., 
attending classes and also practicing online), by completing the course and tester 
contents autonomously while being monitored by a tutor, or following the course 
in a completely autonomous way (both by paying attention to the automatic 
feedback reports that show their results and by comparing their answers with 
the models provided in those cases where automatic correction is not possible). 

The self-assessment modality enables students to conduct their own learning 
process and to assess their own learning achievements in an independent, 
autonomous and individual way. The tutor assessment fosters the development 
of the students’ autonomy and sense of responsibility while enabling them to get 
as much help and support from a human tutor as they need. Both modalities have 
the common goal of making students think critically about themselves and to 
enable them to make decisions and judgements about their own progress, in spite 
of the fact that this is done in different ways depending on the modality chosen. 

The criteria and the objectives of these materials, as well as the contents and the 
way these are organised, have been determined by the guidelines of the Action 
Plan on Language Learning and Linguistic Diversity of the European Union as 
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well as by the characteristics of the FCE. In this way, the FCE Online Course 
and Tester are intended to make students develop their communicative abilities 
in English. This is addressing the necessity that European universities set up a 
clear and coherent linguistic policy that takes into account linguistic and cultural 
diversity, as established by the European Commission, while pursuing the goal 
of helping students acquire a B2 level of English. This level enables them to pass 
the FCE exam, which is a prerequisite for them to be able to graduate from UPV. 

The CAMILLE Research Group will continue working in order to improve 
its FCE preparatory course and tester in terms of quality, user-friendliness, 
efficiency, and usefulness. Trying to foster the students’ autonomy and to prepare 
them better for a global society in which the ability to communicate and to 
interact fluently and spontaneously with speakers of other languages, especially 
in English, is becoming more and more important. 
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